Parallelizing Design of Application Tailored Neural Networks

In a companion paper, a constructive approach for designing feedforward neural networks using genetic algorithms is proposed [7, 8]. The algorithm constructs networks with close to optimum size growing hidden layer units in a problem specific manner and has very good generalization properties. In this paper, in order to make the constructive design algorithm computationally efficient, a two stage speed up method is proposed: (1) parallel genetic search for hidden layer units construction; and (2) the dynamic pocket algorithm for learning the hidden to output layer weights. The proposed parallel method achieves significant computational speed-up over the sequential method and is suitable for distributed implementation. In addition, the dynamic pocket algorithm can be used to speed up various other neural network constructive design methods.